[DOCS-10653] Obs Pipelines OCSF Custom Configuration #28892
Conversation
Co-authored-by: cecilia saixue watt <[email protected]>
Preview links (active after the
|
content/en/observability_pipelines/processors/remap_ocsf/custom_configuration.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_configuration.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_configuration.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_configuration.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_configuration.md
Outdated
Show resolved
Hide resolved
| All enumerated name or label fields identified in the OCSF schema are converted to their sibling `id` field. For example, the string field `severity` is automatically converted to the numeric field `severity_id` based on the standard enum value table defined in the OCSF schema. If no matching value is found in the lookup table, the `id` field is set to `99` to represent `Other`. | ||
|
|
||
| If one of the listed `profiles` in the metadata section is `datetime`, the mapping | ||
| automatically has all numeric timestamps identified in the OCSF schema converted into the sibling field `{DEST}_dt`. For example, the numeric `time` field is converted into `time_dt`, which contains a string representation of that timestamp. No additional work is required to support the `datetime` profile. |
There was a problem hiding this comment.
Is there a link to the OCSF schema we can add here?
There was a problem hiding this comment.
OCSF is composed of multiple schemas, the choice of which is specified by the version, class, and profiles given in the metadata section. The complete schema can be browsed here: https://schema.ocsf.io/
content/en/observability_pipelines/processors/remap_ocsf/custom_configuration.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_configuration.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_configuration.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_configuration.md
Outdated
Show resolved
Hide resolved
This reverts commit f418d94.
…m_mapping_configuration_format.md Co-authored-by: DeForest Richards <[email protected]>
| - type | ||
| - subtype | ||
| - null | ||
| - 'wildcard*' |
There was a problem hiding this comment.
@bruceg is there a reason why 'wildcard*' is in quotes here?
There was a problem hiding this comment.
I quoted it in the belief that the YAML parser might interpret it differently without the quotes. That is, they should all be strings except for the null values, but the auto-quoting would handle the others and I just wasn't sure about this one.
There was a problem hiding this comment.
I tried it out here: https://yaml-online-parser.appspot.com/
And it seems like wildcard* does get auto-quoted without me having to add the quotes, but we can leave it if it's better to have it
There was a problem hiding this comment.
If it is auto-quoted, then I think we should unify the formatting for simplicity.
|
Thanks @drichards-87 ! I applied all of your suggestions and also clarified one of the sections you had a question about. |
content/en/observability_pipelines/processors/remap_ocsf/custom_mapping_configuration_format.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_mapping_configuration_format.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_mapping_configuration_format.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_mapping_configuration_format.md
Outdated
Show resolved
Hide resolved
content/en/observability_pipelines/processors/remap_ocsf/custom_mapping_configuration_format.md
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
All this "Import configuration file" feature is only partially supported in the UI right now: only the mapping field and profiles fields are supported, which means the JSON will look like
{
"mapping":[...]
"profiles":[...]
}
but this doc is exactly what the final iteration will look like in the UI. Can we potentially wait before releasing this doc? (this will arrive sometimes during Q2)
Co-authored-by: Bruce Guenter <[email protected]>
content/en/observability_pipelines/processors/remap_ocsf/custom_mapping_configuration_format.md
Outdated
Show resolved
Hide resolved
What does this PR do? What is the motivation?
Merge instructions
Merge readiness:
For Datadog employees:
Merge queue is enabled in this repo. Your branch name MUST follow the
<name>/<description>convention and include the forward slash (/). Without this format, your pull request will not pass in CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.If your branch doesn't follow this format, rename it or create a new branch and PR.
To have your PR automatically merged after it receives the required reviews, add the following PR comment:
Additional notes